Test Plan
Requested by: Dr. Darren Lim
Assistant Professor
Department of Computer Science
Siena College
Mrs. White
Visiting Professor
Department of Computer Science
Siena College
Generation Java
ID -10t Consultants
Prepared by: Christine Fox
Nick Miller
Kevin Mulcahy
Jordan Steans
February 10th, 2009
Section 1: Introduction
1.1: Introduction & Test Explanation
This document is the first of many possible versions that will be used to describe the testing requirements for the Generation Java. These are the requirements that need to be tested in order for the Generation Java to be considered fully functional. As the project development continues towards a completed project, the requirements will be fulfilled, and some items maybe removed, or revised. This document will keep track of these changes, as well as those requirements that have been completed. Many updates will be made to this document in order to better prepare for the Acceptance Test document due in April 2009. An acceptance test is black-box testing performed on a system prior to its delivery. Generally involves running a suite of tests on the completed system. Each individual test, known as a case, exercises a particular operating condition of the user's environment or feature of the system, and will result in a pass or fail boolean outcome. The acceptance test suite is run against the supplied input data or using an acceptance test script to direct the testers.
The test plan will begin with system testing. System testing consists of software or hardware testing conducted on a complete, integrated system to evaluate the system's compliance with its specific requirements. System testing falls within the scope of black-box testing, and as such, should require no knowledge of the inner design of the code or logic. In system testing, our team is concerned with validating the specified functional requirements on the Generation Java system. The functional requirements of the Generation Java system will be tested in the system test along with non-functional requirements, exception handling, our approach to testing, and will end with the systemŐs pass/fail criteria. This test is a crucial part of the test plan because it will help us determine if we met all of the requirements promised at the beginning of our work. Unit testing is where our team tests each unit individually to make sure that specific unit is free of data, logic, or standard errors. The units have been divided by screen in the system. Every screen in the web system has its own unit and within each unit are a number of test cases. These test cases test specific actions within each unit. The following units and test cases are represented in an Excel table format and as we progress and begin to produce our software, these units may become more in depth and more dense. The final portion of this document contains the integration and regression testing. Regression testing is any type of software testing which seeks to uncover software regressions. Such regressions occur when software functionality that was previously working correctly, stops working as intended. Typically regressions occur as an unintended consequence of program changes. Common methods of regression testing include re-running previously run tests and checking whether previously fixed faults have re-emerged. This is a crucial stage in the testing process because this is where the separate units tested before combine to become a whole. We are concerned with the functionality of the units together and how they interact with each other in this stage of testing. Finally our testing is complete with regression testing which is only incorporated if software functionality that was working normal, stops working completely or no longer works correctly. Regression testing is used to discover regression bugs which may be causing these errors. As the project continues to grow, so too will this test document.
Section 2: System Test
2.1: Test Items & Functional Requirements Inventory
Functional Requirements
Requirements grouped according to Use Case.
System Administrator:
There is only one System Administrator.
Once logged in, a System Administrator has three views available:
á Student YES NO
á Instructor YES NO
á Administrator YES NO
The System Administrator can perform different tasks based on which view is currently be used:
Student View
á Registration
á The System Administrator views the system as a Student that is registered in all possible courses.
YES NO
á The System Administrator is not seen as registered for any course by any user.
YES NO
á The System Administrator does not need to log out to switch between courses.
YES NO
á Questions/Question Sets
á The System Administrator is able to attempt any of the question sets, including all homework, quiz, test, and practice sets.
YES NO
á The System Administrator is able to save code to a database.
YES NO
á Once code is compiled and executed, the System Administrator will receive feedback on it.
YES NO
á The feedback received by the System Administrator is identical to that of a Student; however, it is not viewable by any Instructor.
YES NO
á The System Administrator should be aware when the maximum number of attempts (if specified) has been reached on a question, but should not be restricted by it as a Student would be.
YES NO
á The System Administrator has the ability to click any links (if provided and enabled) on any question.
YES NO
á The System Administrator can view all files posted by all Instructors.
YES NO
Instructor View
á Registration
á The System Administrator views the system as an Instructor that is instructing all possible courses.
YES NO
á The System Administrator is not seen as an Instructor of any course by any user.
YES NO
á The System Administrator does not need to log out to switch between courses.
YES NO
á Questions/Question Sets
á The System Administrator can create, edit, delete, and distribute all questions and question sets.
YES NO
á The System Administrator can set a time and date for any question set to become available.
YES NO
á The System Administrator can set a time and date for any questions set to be due.
YES NO
á The System Administrator can assign point totals to any question set.
YES NO
á Instructor Privileges
á The System Administrator can see all StudentsŐ last submissions and submission information for every (attempted) question in every question set.
YES NO
Administrator View
á The System Administrator can create, edit, and delete the following types of accounts:
á Program Administrator
YES NO
á Instructor
YES NO
á Student
YES NO
á The System Administrator can set a timeout length for all sessions.
YES NO
á The System Administrator can add questions/question sets to the global pool.
YES NO
á The System Administrator may logout in any view.
YES NO
Program Administrator:
A Program Administrator coordinates one more sections/courses.
There may be multiple Program Administrators.
Program Administrators do not register.
A Program AdministratorŐs account is created by the System Administrator.
Once logged in, a Program Administrator has three views available:
á Student YES NO
á Instructor YES NO
á Program Administrator YES NO
A Program Administrator can perform different tasks based on which view is currently be used:
Student View
á Registration
á A Program Administrator views the system as a Student that is registered in all courses specified by the System Administrator.
YES NO
á A Program Administrator is not seen as registered for any course by any user except by the System Administrator.
YES NO
á A Program Administrator does not need to log out to switch between courses.
YES NO
á Questions/Question Sets
á A Program Administrator is able to attempt any of the question sets specified by the System Administrator, including homework, quiz, test, and practice sets.
YES NO
á A Program Administrator is able to save code to a database.
YES NO
á Once code is compiled and executed, A Program Administrator will receive feedback on it.
YES NO
á The feedback received by a Program Administrator is identical to that of a Student; however, it cannot be viewed by any Instructor.
YES NO
á A Program AdministratorŐs grades are not counted in any reports, including reports run by an Administrator.
YES NO
á A Program Administrator should be aware when the maximum number of attempts (if specified) has been reached on a question, but should not be restricted by it as a Student would be.
YES NO
á A Program Administrator has the ability to click any links (if provided and enabled) on any question specified by the System Administrator.
YES NO
á A Program Administrator can view all files posted by Instructors specified by the System Administrator.
YES NO
Instructor View
á Registration
á A Program Administrator views the system as an Instructor that is instructing all courses specified by the System Administrator.
YES NO
á A Program Administrator is not seen as an Instructor of any course by any user.
YES NO
á A Program Administrator does not need to log out to switch between courses.
YES NO
á Questions/Question Sets
á A Program Administrator can create, edit, delete, and distribute course-specific questions and question sets.
YES NO
á A Program Administrator can set a time and date for any question set specified by the System Administrator to become available.
YES NO
á A Program Administrator can set a time and date for any questions set specified by the System Administrator to be due.
YES NO
á A Program Administrator can assign point totals to any question set specified by the System Administrator.
YES NO
á Instructor Privileges
á A Program Administrator can see all StudentsŐ (specified by the System Administrator) last submissions and submission information for every (attempted) question in every question set.
YES NO
á A Program Administrator can see all StudentsŐ (specified by the System Administrator) statistics (including login information) and all statistics for every question in every question set (also specified by the System Administrator).
YES NO
Administrator View
á A Program Administrator can create, edit, and delete the following types of accounts:
o Instructor YES NO
o Student YES NO
á The Program Administrator can add question/question sets to any of the course-specific pools specified by the System Administrator.
YES NO
á The Program Administrator may logout in any view.
YES NO
Instructor:
An Instructor does not register.
An InstructorŐs account is created by a Program Administrator or the System Administrator.
Once logged in, an Instructor has two views available:
á Student YES NO
á Instructor YES NO
An Instructor can perform different tasks based on which view is currently be used:
Student View
á Registration
á An Instructor views the system as a Student that is registered in all courses specified by the Program Administrator.
YES NO
á An Instructor is not seen as registered for any course by any user.
YES NO
á An Instructor does not need to log out to switch between courses.
YES NO
á Questions/Question Sets
á An Instructor is able to attempt any of the question sets specified by the Program Administrator, including homework, quiz, test, and practice sets.
YES NO
á An Instructor is able to save code to a database.
YES NO
á Once code is compiled and executed, an Instructor will receive feedback on it.
YES NO
á The feedback received by an Instructor is identical to that of a Student. No one else can view this feedback.
YES NO
á An InstructorŐs grades are not counted in any reports, even reports run by an Administrator.
YES NO
á An Instructor should be aware when the maximum number of attempts (if specified) has been reached on a question, but should not be restricted by it as a Student would be.
YES NO
á An Instructor has the ability to click any links (if provided and enabled) on any question specified by the Program Administrator.
YES NO
á An Instructor can view all files posted by themselves or other Instructors specified by the Program Administrator.
YES NO
Instructor View
á Registration
á An Instructor can view all courses specified by a Program Administrator.
YES NO
á An Instructor is seen as the Instructor for all courses specified by the Program Administrator.
YES NO
á An Instructor does not need to log out to switch between courses.
YES NO
á If an Instructor wishes to teach a new course, the new course may be added by inputting the Course ID into the appropriate location. A Course ID is provided by the Program Administrator or System Administrator.
YES NO
á Questions/Question Sets
á An Instructor has the ability to create, edit, and delete java questions in the InstructorŐs own private pool.
YES NO
á An Instructor can upload a question or enter it into a text box.
YES NO
á An Instructor may provide links to aid a Student.
YES NO
á An Instructor has the ability to create, edit and delete question sets in the InstructorŐs own private pool.
YES NO
á An Instructor can use questions from the InstructorŐs private pool, the course-specific pool, or the global pool when creating question sets.
YES NO
á When creating a question set, an Instructor can set point totals and a maximum number of attempts allowed for each question.
YES NO
á An Instructor can set a time and date for a question set to become available and unavailable.
YES NO
á If an Instructor wishes to add a question or question set to the course-specific pool, it must be submitted to the Program Administrator.
YES NO
á If an Instructor wishes to add a question or question set to global pool, it must be submitted to the System Administrator.
YES NO
á Instructor Privileges
á An Instructor can see all StudentsŐ (registered in the InstructorŐs course) last submissions and submission information for every (attempted) question in every question set.
YES NO
á An Instructor may edit a StudentŐs grade as seen fit.
YES NO
á An Instructor may deny a student access to a course.
YES NO
á An Instructor may post files that can be viewed by Students in the InstructorŐs course.
YES NO
An Instructor can log out in either view.
YES NO
Student:
A Student has only one view.
á Registration
á A Student must register in a specific course by entering a Course ID in the appropriate location. (A Course ID is provided by the Instructor of that course)
YES NO
á A Student may be registered in multiple courses under the same username and password.
YES NO
á A Student does not need to log out in order to change which course is currently being worked in.
YES NO
á Question Sets
á A Student has the ability to view and attempt any question set provided by the Instructor.
YES NO
á When a Student submits code, feedback is returned indicating the codeŐs degree of correctness.
YES NO
á A Student can retry a question until the number of attempts (if specified) has run out.
YES NO
á A Student may see only their own grades and statistics.
YES NO
á Additional Privileges/Restrictions
á A Student may view any link or file provided by the Instructor (if enabled).
YES NO
á A Student may not click out of the testing window during a test or the test will be terminated.
YES NO
á A Student may log out as desired.
YES NO
2.2: Non-Functional Requirements Inventory Testing
* The system should be quick and smooth, using fast servers and databases.
* The system should also be aesthetically pleasing.
2.3: Exception Handling
Exception handling is a method used by the system to detect and recover from exceptional conditions. An exceptional condition is any unexpected occurrence that cannot be accounted for in a systemŐs normal operation. The developers of Generation Java realize that it can be difficult to protect a system from the effects of exceptional conditions because, by nature, they are unusual occurrences that can generally not be anticipated. Therefore, the developers of Generation Java will implement programmed and default exception handlers that will attempt to recover the system from any unanticipated conditions.
Programmed exception handling modules are mechanisms built into software for specific exceptional cases that are known and are likely to occur. Since these occurrences are relatively well understood, protection for them will be incorporated into Generation Java. For example, if the User does not provide all of the required information for registration, then the system will prompt them to resubmit their information again with the proper data.
For all exceptional conditions that are not anticipated by the system developers, a default exception handler will be built in. The default exception handler must catch-all unexpected exceptions and return the system to a normal state of operation.
2.4 Approach to Testing
Software testing is an activity aimed at verifying that a correct solution to the problem has been developed such that it satisfies the functional and non-functional requirements.
In general, most of the outlined in this document will be unit and integration testing. System-level testing will also be carried out and documented on the components once they have been fully developed.
It is also important to note that in general, our testing will be structured to evaluate correctness of the code. It is anticipated that since the future phases of code development will be focused on performance enhancement, performance of the code (a non-functional requirement) will then be monitored closely. Consequently, routine performance testing should be unnecessary, while testing for correctness will be important as changes are made to speed up the code.
Section 4: Integration Test
4.1: Explanation & Integrated Units
The International Online Survey is being created on the foundation of already produced modules, the integration and regression tests conducted for this project will consist mostly of user created instances.
In the event that a unit does not pass the following tests, that unit will be fixed, and then that unit along with all related units will be re-evaluated for accuracy. In the event that one unit passes and the other does not pass, both units will need to be evaluated in order to determine where and why the error occurred. In that event, depending on the circumstances in which the error occurred, all affected units may need to be retested.
|
Unit Number |
Linked Units |
|
1 |
2, 4, 6, 7, 8, 10 |
|
2 |
2 |
|
3 |
32 |
|
4 |
33, 39 |
|
5 |
33, 40 |
|
6 |
33 |
|
7 |
9 |
|
8 |
3 |
|
9 |
4, 10 |
|
10 |
9 |
|
11 |
24, 33 |
|
12 |
36, 37 |
|
13 |
32 |
|
14 |
33 |
|
15 |
33 |
|
16 |
16 |
|
17 |
15, 38 |
|
18 |
19, 20, 21 |
|
19 |
18, 20, 21 |
|
20 |
18, 19, 21 |
|
21 |
18, 19, 20 |
|
22 |
23 ,25 |
|
23 |
23 |
|
24 |
4, 5, 6, 11 |
|
25 |
4, 5, 6, 17 |
|
26 |
18, 34, 35 |
|
27 |
26, 35 |
|
28 |
32 |
|
29 |
6 |
|
30 |
5 |
|
31 |
4 |
|
Unit Number |
Linked Units |
|
32 |
12 |
|
33 |
12 |
|
34 |
6 |
|
35 |
6 |
|
36 |
12 |
|
37 |
12 |
|
38 |
4, 17 |
|
39 |
4 |
|
40 |
5 |